Methods and systems for crop pest management utilizing geospatial images and microclimate data

ABSTRACT

Systems and methods for predicting pest susceptibility, comprising steps to receive geocoded geospatial image data of a crop field from sensors, receive microclimate data of the crop field, determine a crop vigor map for the crop field, and then generate a pest susceptibility map utilizing a risk model based on the crop vigor map and the microclimate data. The pest susceptibility map comprises a measure of a susceptibility of a crop in the crop field to one or more crop pests at one or more locations. In some embodiments, the method also comprises steps to generate a treatment plan (e.g., pesticide application) and to estimate an anticipated return on investment (ROI). The system therefore leverages remote-sensed data, machine data, analytics, and machine learning to enable farmers to predict, prevent, and control the outbreak of crop pests to greatest economic effect. Such a system addresses a fundamental problem in agriculture.

REFERENCE TO RELATED APPLICATIONS

This application is a non-provisional of and claims priority from U.S.Ser. No. 62/849,779, filed on 17 May 2019, entitled “METHODS AND SYSTEMSFOR CROP DISEASE MANAGEMENT UTILIZING MACHINE DATA AND REMOTE SENSING,”the entire disclosure of which is hereby incorporated by reference inits entirety herein.

This application is also related to U.S. Pat. No. 10,143,148, issued on4 Dec. 2018, filed as U.S. Ser. No. 15/154,926, filed on 13 May 2016,entitled “METHODS AND DEVICES FOR ASSESSING A FIELD OF PLANTS FORIRRIGATION,” the entire disclosure of which is hereby incorporated byreference in its entirety herein.

NOTICE OF COPYRIGHTS AND TRADEDRESS

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. This patent document may showand/or describe matter which is or may become tradedress of the owner.The copyright and tradedress owner has no objection to the facsimilereproduction by anyone of the patent disclosure as it appears in theU.S. Patent and Trademark Office files or records, but otherwisereserves all copyright and tradedress rights whatsoever.

FIELD OF THE INVENTION

Embodiments of the present invention are in the field of automated cropmeasurements and pertain particularly to determinations of conditionswhich may be amenable to pest outbreaks in crop fields based upongeospatial image data and microclimate data.

BACKGROUND OF THE INVENTION

The statements in this section may serve as a background to helpunderstand the invention and its application and uses, but may notconstitute prior art.

Scleroninia stem rot, also known as white mold, is among the top 10diseases plaguing soybeans, and can cause major yield (and correspondingeconomic) losses. For example, in 2014, white mold triggered yieldlosses resulting in approximately $500 million in economic losses bysoybean farmers in the U.S. White mold is primarily controlled throughthe application of fungicides and occasionally herbicides, especiallyduring the period of soybean flowering (R1 to R3), as this is the periodwhen the flowers are vulnerable to ascospores (produced from S.sclerotiorum apothecia) and environmental conditions are amenable towhite mold development.

White mold only develops when soybean flowering, apothecial germination,and weather conditions that are conducive for infection occursimultaneously (See, Weather-Based Models for Assessing the Risk ofSclerotinia sclerotiorum Apothecial Presence in Soybean (Glycine max)Fields, November 2017, Available athttps://apsjournals.apsnet.org/doi/10.1094/PDIS-04-17-0504-RE.)Therefore, predicting white mold outbreaks from measured data would behighly advantageous.

More generally, managing agricultural inputs precisely, includingchemicals to manage disease, is important for maximizing yieldproductivity while minimizing costs in some embodiments. Insufficient ormis-timed applications can lead to major yield losses and even cropdeath. However, unnecessary applications impose a large cost without anycorresponding benefit and may have additional negative side effects.Understanding where and when to apply chemicals to manage disease istherefore a matter of high importance for farmers.

Crop pest management can be made more efficient by identifying whichfields (or portions of fields) are likely to be prone to crop pestoutbreak, including crop diseases, insects, weeds, and plant pathogens.Current practice involves time-intensive direct assessments of thefield(s) to identify the presence of disease (at which point it may betoo late to be treated effectively) or to make field observationssuggesting a potential disease outbreak based upon the experience of theobserver or other available heuristics. Such practices lack precisionand may result in fields being treated when they should not be, fieldsnot being treated when they should, or portions of fields either beingtreated or not without sound information on the presence or potentialfor an actual pest outbreak in the relevant area.

Even if growers or their advisors attempt to employ more predictiveapproaches, they are required to estimate certain measures (e.g., cropcanopy closure) and to evaluate each field based upon specific criteria(e.g., row spacing). Such estimations lack accuracy as it is difficult,if not impossible, to obtain a reliable measure of canopy closure basedupon ground-level observations. Furthermore, growers may lack records ofkey inputs (e.g., row spacing) or may enter any such inputs incorrectly,resulting in an inaccurate prediction.

The aforementioned difficulties result in unsatisfactory experiences forfarmers as well as difficulties with crop pest management. The presentinvention was developed to address these and other difficulties, and toprovide an automated method for determining when a pest outbreak isprobable based upon specific and quantitative measures. Thus theinvention enables farmers to better manage crop pests throughapplications of agricultural chemicals or other management techniques.

It is against this background that various embodiments of the presentinvention were developed.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention relate to automated cropmeasurements and pertain particularly to predication of pestsusceptibility in crop fields based upon crop vigor maps determined fromgeospatial image data (e.g., aerial/satellite imagery), and microclimatedata (i.e., locally variable environmental conditions such astemperature and humidity).

The following presents a summary to provide a basic understanding of oneor more embodiments of the disclosure. This summary is not intended toidentify key or critical elements, or to delineate any scope ofparticular embodiments or any scope of the claims. Its sole purpose isto present concepts in a simplified form as a prelude to the moredetailed description that is presented later.

In one or more embodiments described herein, systems,computer-implemented methods, apparatuses and/or computer programproducts provide an automated method for determining when a pestoutbreak in particular crop field(s) or portions thereof is probablebased upon specific, quantitative measures. The system may include anon-transitory, computer-readable medium containing instructions and aprocessor that executes the instructions to perform various stages.

The present invention is directed to methods and systems for assessingfield(s) of plants to determine, in a specific and quantitative mannerbased upon data and measurements, which field(s) or portions thereofexhibit characteristics corresponding to likely pest outbreak, and thusare good candidates for applications of agricultural chemicals or othermanagement techniques to prevent the outbreak, or control thepropagation, of crop pest(s). The primary users of this data areagricultural growers or their advisors (e.g., agronomists), who are ableto receive data-driven assessments of which field(s) or portions thereofexhibit the conditions necessary for the outbreak of disease(s).

In short, the inventors have developed a process to predict where andwhen to employ agricultural management techniques (e.g., applications ofspecific chemicals such as fungicides) to the greatest effect to preventthe outbreak, or control the propagation, of crop pest(s). Oneapplication of this process is to white mold infestations in soybeans inthe central and upper reaches of the United States Corn Belt.

Accordingly, one embodiment of the present invention is a systemcomprising a hardware processor and a non-transitory storage medium forstoring program code, the program code executable by the hardwareprocessor to execute a process for predicting a pest susceptibility, theprogram code when executed by the hardware processor causing thehardware processor to execute steps to: receive geospatial image data ofa crop field from one or more sensors, wherein the geospatial image dataof the crop field is geocoded by longitude and latitude coordinates;receive microclimate data of the crop field, wherein the microclimatedata comprises locally variable environmental conditions in one or moreportions of the crop field; determine a crop vigor map for the cropfield from the geospatial image data, wherein the crop vigor mapcomprises a crop vigor index on the one or more portions of the cropfield, and wherein the crop vigor index is a numerical index based on afoliage volume, density, layout, and/or health status of the crop in thecrop field; and generate a pest susceptibility map utilizing a riskmodel based on the crop vigor map and the microclimate data, wherein thepest susceptibility map comprises a pest susceptibility index geocodedby latitude and longitude coordinates, and wherein the pestsusceptibility index is a measure of a susceptibility of a crop in thecrop field to one or more crop pests.

In one embodiment, the determining the crop vigor map for the crop fieldcomprises calculating the crop vigor index from one or more crop vigorindicator equations using the geospatial image data as input.

In one embodiment, the determining the crop vigor map for the crop fieldutilizes a first machine vision algorithm executable by the hardwareprocessor.

In one embodiment of this aspect, the machine vision algorithm comprisesone or more deep learning neural networks, wherein the deep learningneural networks are trained on ground truth data comprising geospatialimage data of one or more sample crop fields and one or more crop vigormaps for the one or more sample crop fields.

In one embodiment, the system further comprises program code to receiveone or more crop field parameters, wherein the one or more crop fieldparameters are used in combination with the crop vigor map in the riskmodel to predict the pest susceptibility.

In one embodiment of this aspect, the one or more crop field parametersare determined from the geospatial image data utilizing a second machinevision algorithm executable by the hardware processor.

In another embodiment of this aspect, the one or more crop fieldparameters are machine data received from agricultural equipment.

In another embodiment of this aspect, the one or more crop fieldparameters comprise crop row spacing of the crop field, wherein the croprow spacing is estimated from the geospatial image data using a thirdmachine vision algorithm and/or received from machine data from one ormore agricultural equipment.

In another embodiment of this aspect, the one or more crop fieldparameters comprise an irrigation status of the crop field, wherein theirrigation status is estimated using geospatial image data and/orreceived from machine data from one or more irrigation equipment.

In another embodiment of this aspect, the one or more crop fieldparameters comprise a degree of canopy closure of the crop field,wherein the degree of canopy closure is estimated from the geospatialimage data using the first machine vision algorithm.

In yet another embodiment of this aspect, the one or more crop fieldparameters comprise a crop stage of the crop field, wherein the cropstage is determined from physical observation and/or one or more cropmodels.

In one embodiment of the invention, the risk model is a machine learningalgorithm executable by the hardware processor, wherein the machinelearning algorithm is trained on ground truth data comprising one ormore sample pest data points and one or more sample crop vigor maps forone or more sample crop fields.

In one embodiment of this aspect, the machine learning algorithmcomprises one or more of a linear regressor, a nonlinear regressor, arandom forest algorithm, and a neural network.

In one embodiment of the invention, the crop pest is selected from thegroup consisting of crop diseases, insects, weeds, and plant pathogens.

In another embodiment, the geospatial image data is selected from thegroup consisting of aerial imagery, satellite imagery, and unmannedaircraft system (UAS) imagery, wherein the one or more sensors areinfrared cameras.

In another embodiment, the one or more sensors are located on a machineselected from the group consisting of an unmanned aerial vehicle (UAV),an unmanned aircraft system (UAS), an aircraft, a satellite, and a fieldequipment.

In another embodiment, the microclimate data comprises a microclimatemap, wherein the microclimate map is generated from the group consistingof one or more in-field measurements, one or more aerial measurements,one or more satellite measurements, one or more drone measurements, andone or more microclimate models.

In another embodiment, the system further comprises program code togenerate a treatment plan based on the pest susceptibility map, whereinthe treatment plan comprises an agricultural management technique,comprising application of one or more agricultural chemicals, to preventoutbreak or control propagation of one or more crop pests.

In yet another embodiment, the system further comprises program code toreceive price information for a crop growing in the crop field, a costinformation for one or more agricultural management techniques, and ananticipated efficacy for the one or more agricultural managementtechniques; and generate an anticipated return on investment (ROI) basedon the price and cost information and the anticipated efficacy.

Another embodiment of the present invention is a non-transitory storagemedium for storing program code, the program code executable by ahardware processor to execute a process for predicting a pestsusceptibility of a crop field, the program code when executed by thehardware processor causing the hardware processor to execute theaforementioned steps.

Yet another embodiment of the present invention is acomputer-implemented method for predicting a pest susceptibility, thecomputer-implemented method executable by a hardware processor, themethod comprising: receiving geospatial image data of a crop field fromone or more sensors, wherein the geospatial image data of the crop fieldis geocoded by longitude and latitude coordinates; receivingmicroclimate data of the crop field, wherein the microclimate datacomprises locally variable environmental conditions in one or moreportions of the crop field; determining a crop vigor map for the cropfield from the geospatial image data, wherein the crop vigor mapcomprises a crop vigor index on the one or more portions of the cropfield, and wherein the crop vigor index is a numerical index calculatedbased on a foliage volume, density, layout, and/or health status of thecrop in the crop field; and generating a pest susceptibility maputilizing a risk model based on the crop vigor map and the microclimatedata, wherein the pest susceptibility map comprises a pestsusceptibility index geocoded by latitude and longitude coordinates, andwherein the pest susceptibility index is a measure of a susceptibilityof a crop in the crop field to one or more crop pests.

Yet other aspects of the present invention include methods, processes,and algorithms comprising the steps described herein, and also includethe processes and modes of operation of the systems and serversdescribed herein. Yet other aspects and embodiments of the presentinvention will become apparent from the detailed description of theinvention when read in conjunction with the attached drawings.

Features from any of the embodiments described herein may be used incombination with one another in accordance with the general principlesdescribed herein. The details of one or more embodiments of the subjectmatter described herein are set forth in the accompanying drawings andthe description below. These and other aspects of the invention willbecome apparent from the following description of the preferredembodiments, drawings, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention described herein are exemplary, andnot restrictive. Embodiments will now be described, by way of examples,with reference to the accompanying drawings, in which:

FIG. 1A shows an illustrative flowchart of a process for pestsusceptibility prediction, in accordance with one embodiment of thepresent invention.

FIG. 1B shows another illustrative flowchart of a process for pestsusceptibility prediction, in accordance with another embodiment of thepresent invention.

FIG. 2 shows an illustrative block diagram of a process to predict apest susceptibility of a crop field, in accordance with yet anotherembodiment of the present invention.

FIG. 3 shows an illustrative flowchart for a process for the generationof a treatment map for a crop field, in accordance with yet anotherembodiment of the present invention.

FIG. 4 shows an illustrative flowchart of a process to determine whichfields, or portions thereof, exhibit conditions amenable to pest ordisease outbreak and/or propagation, in accordance with yet anotherembodiment of the present invention.

FIG. 5 is an exemplary schematic diagram of a server (managementcomputing entity) for implementing the present invention, in accordancewith example embodiments of the disclosure.

FIG. 6 is an exemplary schematic diagram of a client (user computingentity) for implementing the present invention, in accordance withexample embodiments of the disclosure.

FIG. 7 shows an illustrative block diagram of a convolutional neuralnetwork (CNN) for geospatial image data analysis using a machine visionmodule, in accordance with one embodiment of the invention.

FIG. 8 shows an illustrative block diagram for a machine learningalgorithm using random forest regressors, in accordance with anotherembodiment of the invention.

FIG. 9 shows an example flow diagram for training the machine learningalgorithm, in accordance with example embodiments of the disclosure.

FIG. 10 shows an illustrative block diagram showing functionalitiesprovided by an extended CERES platform, according to one embodiment ofthe present invention.

FIG. 11 shows an exemplary regression relationship between canopytemperature and crop vigor index data points, according to oneembodiment of the present invention.

FIG. 12A shows canopy temperatures of an illustrative sugarbeet field,as computed by an aerial thermal camera, according to one embodiment ofthe present invention.

FIG. 12B shows a Chlorophyll vegetation index map (i.e., a crop vigormap) calculated for the same illustrative crop field and on the sameflight as FIG. 12A.

FIG. 13A shows an illustrative example of a crop vigor map with anoverlay of sample measured pest (white mold) index data points,according to one embodiment of the present invention.

FIG. 13B shows a predicted pest susceptibility map for the example cropfield of FIG. 13A, according to one embodiment of the present invention.

Throughout the drawings, identical reference characters and descriptionsindicate similar, but not necessarily identical, elements. While theexemplary embodiments described herein are susceptible to variousmodifications and alternative forms, specific embodiments have beenshown by way of example in the drawings and will be described in detailherein. However, the exemplary embodiments described herein are notintended to be limited to the particular forms disclosed. Rather, thepresent disclosure covers all modifications, equivalents, andalternatives falling within the scope of the appended claims.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, for purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the invention. It will be apparent, however, to oneskilled in the art that the invention can be practiced without thesespecific details. In other instances, structures, devices, activities,and methods are shown using schematics, use cases, and/or flow diagramsin order to avoid obscuring the invention. Although the followingdescription contains many specifics for the purposes of illustration,anyone skilled in the art will appreciate that many variations and/oralterations to suggested details are within the scope of the presentinvention. Similarly, although many of the features of the presentinvention are described in terms of each other, or in conjunction witheach other, one skilled in the art will appreciate that many of thesefeatures can be provided independently of other features. Accordingly,this description of the invention is set forth without any loss ofgenerality to, and without imposing limitations upon the invention.

Indeed, these disclosures may be embodied in many different forms andshould not be construed as limited to the embodiments set forth herein;rather, these embodiments are provided so that this disclosure willsatisfy applicable legal requirements. The term “or” is used herein inboth the alternative and conjunctive sense, unless otherwise indicated.The terms “illustrative” and “exemplary” are used to refer to exampleswith no indication of quality level. Like numbers refer to like elementsthroughout.

The terms “crop field,” “field,” “plant,” and “field of plant(s)” asused herein are generic to any field, orchard, patch, or othercultivated land, and any type of plant growing thereon.

The terms “CERES” and “CERES IMAGING” are trademark names carryingembodiments of the present invention, and hence, the aforementionedtrademark names may be interchangeably used in the specification anddrawing to refer to the products/services offered by embodiments of thepresent invention. The term CERES may be used in this specification todescribe the overall platform, as well as the company providing saidplatform. With reference to the figures, embodiments of the presentinvention are now described in detail.

Overview

FIGS. 1A and 1B present an overview of the present invention, inaccordance to embodiments of the present invention. FIG. 1A shows anillustrative flowchart of a process for pest susceptibility prediction,in accordance with one embodiment of the present invention. The processbegins at step 101, where the process receives geospatial image data,where geospatial image data is defined as sensor and machine data thatis geocoded according to a coordinate system (e.g., by longitude andlatitude coordinates). In other words, the geospatial image data ismapped to geolocations of the crop field.

The geocoded image data received at step 101 may include aerial imagery,satellite imagery, unmanned aircraft system (UAS), imagery from infraredor visible-band cameras. The sensors providing the geocoded image datamay be located on a machine such as unmanned aerial vehicle (UAV), anunmanned aircraft system (UAS), an aircraft, a satellite, or a fieldequipment.

At step 102, the process determines a crop vigor map from the geospatialimage data, where the crop vigor map comprises at least one crop vigorindex that is mapped to at least one geolocation of the crop field. Thecrop vigor index is a numerical index indicating crop health or densityand may be calculated based on the foliage volume, density, layout,and/or health status of the plants in the crop field.

In one embodiment of the present invention, the crop vigor index isdetermined utilizing a machine vision algorithm executable by a hardwareprocessor, as described in further detail in FIG. 7. In anotherembodiment, the crop vigor index is calculated from one or more cropvigor indicator equations using the geospatial image data and themicroclimate data as input. Crop vigor indicators and microclimate dataare discussed in more detail below.

At step 104, the process receives microclimate data, where microclimatedata comprises locally variable environmental conditions from at leastone geolocation of the crop field.

Microclimate parameters extracted from the microclimate data may includetemperature and humidity, as described in FIG. 2 below. In oneembodiment of the present invention, the microclimate data is amicroclimate map comprising at least one microclimate parameter mappedto at least one geolocation of the crop field.

In one embodiment, the microclimate map may also be generated throughone or more microclimate models. Microclimate models are used tosimulate locally variable environmental conditions when directmeasurements are difficult to obtain. For example, a Euclideandistance-based model is a model that can be used to map parts of a fieldthat are close to a windbreak, leading to cooler and wetter conditionscompared to other parts of the field. Solar radiation exposure-basedmodel is another model that considers topography and sun position toquantify cool and wet shaded slopes versus warm and dry exposed slopesduring critical parts of the growing season. Microclimate parameters arefurther discussed below.

At step 108, the process applies a risk model to generate a pestsusceptibility map based on the received crop vigor map 102 andmicroclimate data 104, where the pest susceptibility map comprises atleast one pest susceptibility index mapped to at least one geolocationof the crop field. The pest susceptibility index is a measure of asusceptibility of a crop in the crop field to one or more crop pests,where crop pests include crop diseases, insects, weeds, plant pathogens,and so one.

In one embodiment, the risk model is a machine learning algorithmexecutable by the hardware processor, as described in further detail inFIG. 8. In some embodiments, the risk model utilizes other crop fieldparameters in addition to the crop vigor map and microclimate data togenerate the pest susceptibility map, as shown in FIG. 1B. Finally, theprocess outputs the predicted pest susceptibility at step 110. The riskmodel is further discussed below.

FIG. 1B shows another illustrative flowchart of a process for pestsusceptibility prediction, in accordance with another embodiment of thepresent invention. The process begins at step 112, where the processreceives geospatial image data. At step 114, the process determines acrop vigor map 116 from the geospatial image data utilizing a firstmachine vision algorithm executable by a hardware processor. At step122, the process receives microclimate data, where microclimate datacomprises locally variable environmental conditions from at least onegeolocation of the crop field. The process extracts microclimateparameters 124 from the microclimate data, as described in FIG. 2.

At step 118, the process optionally estimates one or more crop fieldparameters (e.g., canopy closure) 120 based on the geospatial image datautilizing a second machine vision algorithm executable by the hardwareprocessor. The machine vision algorithms used to determine the cropvigor map and the crop field parameters are described in further detailin FIG. 7. Other crop field parameters 128 may also optionally bereceived by the process as shown in step 126. In one embodiment, theother crop field parameters 128 are machine data received fromagricultural equipment or other equipment in or near the field.

The crop field parameters estimated in step 118 or received in step 126may include the crop row spacing at one or more geolocations of the cropfield, the irrigation status, the degree of canopy closure, or the cropstage, each estimated 120 or received 128 for one or more geolocationsof the crop field. In one embodiment, the irrigation status may beestimated using geospatial image data and/or received from machine datafrom one or more irrigation equipment. In one embodiment, the degree ofcanopy closure may be estimated from the geospatial image data using thesame machine vision algorithm used in step 114. In one embodiment, thecrop stage may be determined from physical observation and/or one ormore crop models, as discussed in further detail below.

At step 130, the process applies a risk model to predict pestsusceptibility based on the crop vigor map 116, the estimated crop fieldparameters 120, the microclimate parameters 124, and the other cropfield parameters 128, where the predicted pest susceptibility comprisesat least one pest susceptibility index of the crop field. Finally, theprocess outputs the predicted pest susceptibility at step 132.

In some embodiments, the process generates a treatment plan (not shownin FIGS. 1A and 1B) based on the pest susceptibility map, as shown inFIG. 2. The treatment plan comprises a recommended agriculturalmanagement technique (e.g., application of one or more agriculturalchemicals) to prevent the outbreak or control the propagation of one ormore crop pests.

In some embodiments, the process further receives price information fora crop growing in the crop field, cost information for one or moreagricultural management techniques, and an anticipated efficacy for theagricultural management techniques. The process then generates ananticipated return on investment (ROI) based on the price and costinformation and the anticipated efficacy, as discussed below.

Risk Model Inputs and Outputs

The risk model (108, 130) in one embodiment has two inputs: a crop vigormap (102, 116) having at least one crop vigor index, and microclimateparameters (124) extracted from microclimate data (104, 122). Multipleways of calculating a crop vigor index exist. Generally, the presentinvention relies on calculations from drone, aerial, satellite, and/ormachine-mounted sensors to determine crop vigor. The sensors used aresimilar to those used in aerial imagery, operating in thermal,multi-spectral, infrared, or visible bands, where sensor data isassigned coordinates (e.g., geocoded). Ground or in-field sensors mayalso be used. Ground sensors are mounted above the canopy, sometimes onthe side of the crop fields (e.g., vineyards), typically mounted on barsor booms directed down on the canopy. Some ground sensors may be denselydistributed over the field (e.g., 4-5 feet away from each other).

Band ratios and equations are typically used to quantify crop vigorindices and vegetation indices. For example, the normalized differencevegetation index (NDVI), a vegetation index equation, may be used. TheNDVI index is based on the premise that healthy plants exhibit highreflectance to near infrared (NIR) bands and low reflectance to red(RED) bands. Consequently, a low NDVI indicates a stressed crop whereasa high NDVI indicates a vigorous crop. Another equation is thenormalized difference red edge (NDRE) index, an index similar to theNDVI but using the red edge (RE) part of the spectrum. Next-generationcanopy vigor indices use novel bands/equations (e.g., modifiedchlorophyll absorption ratio index 2 (MCARI2)). Some embodiments of thecurrent invention use deep learning techniques to produce improvedaccuracy.

The second input to the risk model (108, 130) is microclimate data (104,122). The local microclimate in individual fields varies widelydepending on the climate and on crop conditions (e.g., canopy density).Areas with thick, dense, or tall canopy are cooler during the day,warmer at night, experience less sunlight, are more likely to be humidor wet, and receive less exposure to wind. Likewise, areas with light,open, or short canopy tend to be warmer during the day, cooler at night,experience more exposure from sunlight, are less likely to be humid orwet, and are significantly more likely to be windy. All of thesemicroclimate characteristics have a direct influence on the likelihoodof pest outbreak. Microclimate data (e.g., air temperature, relativehumidity, wind speed, radiation, solar insolation, and/or sun exposure)may be obtained through sensor measurements or microclimate models. Inone aspect of the present invention, the microclimate parameters used inthe risk model are measured using sensors placed in or near the field(s)in question. In another aspect, the microclimate parameters areestimated using models or “synthetic sensors.” “Synthetic sensors” areestimates of specific parameters at particular locations based uponinterpolated points within a broader microclimate or weather model.Microclimate parameters may also be obtained from third parties, such asthird-party ground-based or remote weather stations. In yet otherembodiments, microclimate parameters are measured remotely (e.g.,aerially).

In one embodiment, the risk model (108, 130) may use a machine learningmodule, as described below. For example, the machine learning module mayuse an ensemble of random forest regressors, where the predictorvariables are the crop vigor map (102, 116) and other remotely sensed orestimated crop field parameters (120, 128), in conjunction withmacroclimate weather data 122. The dependent variable may be the pest ofinterest (e.g., white mold severity in soybeans). Model training isperformed, where ground truth data comprising the coordinates andcorresponding disease severity from a field at the end of the season maybe used for modeling with in-season crop vigor indices, or otherremotely sensed crop field parameters and macroclimate conditions at thetime of imagery collection. This process quantifies the relationshipbetween pest severity and in-season variables of crop vigor or otherremotely sensed crop parameters and macroclimate conditions. Thisprocedure should be repeated for the pest and crop of interest, asdifferent relationships are expected, resulting in unique crop- andpest-dependent models. It is important to note that training with datafrom coordinates with all pest outbreak levels (e.g., no disease, lightdisease, moderate disease, and severe disease) is preferred. Furtherdetailed discussion of the risk model is given below (e.g., see FIGS. 2,3, 8).

FIG. 2 outlines one embodiment of a process according to the presentdisclosure, which illustrates a process to predict a pest susceptibilityof a crop field. Several inputs and input parameters 210 areautomatically determined from source data 205. The source data 205comprises microclimate data and geospatial image data, such as aerial,satellite, and drone images, as well as sensor and machine datacollected remotely or on the crop field. In addition, source data mayalso include crop stage models. The process generates a set of inputsrelative to crop vigor 210 and a set of input parameters 211 from thesource data 205. The crop vigor inputs 210 are extracted from thegeospatial image data and include a crop vigor index applying to thewhole crop field (i.e., field-scale crop vigor index) or a set ofgeolocated crop vigor indices forming a fine-scale crop vigor map forthe crop field. The input parameters 211 comprise microclimate data(e.g., air temperature, relative humidity, wind speed, solar insolationand/or sun exposure) relative to the crop field. The input parameters211 may also comprise agronomic data (e.g., row spacing, irrigationstatus, crop stage, and canopy closure) at various times (e.g.,planting, treatment application) as well. A risk model 220 is used topredict the susceptibility of an outbreak of one or more crop pests(e.g., white mold in soybeans) based upon the crop vigor 210 and inputparameters 211. An output 230 of the process is a measure of pestsusceptibility at a fine scale (i.e., pest susceptibility map) or at afield scale (e.g., pest susceptibility index). A treatment plan 231 forthe crop field may also be generated from the generated pestsusceptibility, where the treatment plan comprises the implementation ofagricultural management techniques (e.g., applications of specificchemicals such as fungicides) to prevent the outbreak, or control thepropagation, of crop pest(s). In one embodiment, an output treatmentplan 231 may depend on whether the field-scale (i.e., macro) and/or thefine-scale (i.e., micro) conditions for a pest outbreak are met, leadingto (a) full treatment of the crop field if macro conditions are met andmicro conditions are met throughout the field; (b) targeted treatment ofcertain portions of the crop field if macro conditions are not met andmicro conditions are met for those portions, or (c) no treatment ifmacro conditions are not met or micro conditions are not met throughoutthe crop field. The process of FIG. 2 therefore converts the crop vigorindex map and other input parameters (210, 211) into a prescription foran appropriate agricultural management procedure, such as sprayingpesticide (e.g., herbicide, insecticide, fungicide, insect repellent,animal repellent) in parts of the field that are most likely toexperience pest outbreaks. Agricultural management procedures alsoinclude applying fertilizer and plant nutrients. The output pestsusceptibility values 230 can be broken down into zones for targetingdifferent treatments (e.g., spray rates) based on the product beingapplied, the equipment available for application, and the risk agrower/agronomist is willing to take.

In one embodiment, the source data 205 is remote sensed and/or modeledto deduce the agronomic and microclimate parameters 211, and thus makeit possible to predict from remote data alone the likely risk of a pest(e.g., white mold in soybeans) outbreak in a particular crop field orportion of a particular crop field. Therefore, a major advantage of theembodiments described herein is that it is not necessary to measure allinput environmental variables on the ground in order to provide value tothe grower or retailer. Furthermore, owing to the embodiments describedherein, it is not necessary to measure pest severity in all fields, asresults from detailed measurements from sample fields can transferred tofields with similar crops and environmental parameters.

The present invention addresses some of the limitations of the prior artby providing a pathway to use machine data, remote sensing data, and/ormachine learning to determine which fields, or portions thereof, exhibitconditions amenable to pest outbreak or propagation. In another aspect,the present invention provides advantages over prior art methods in thatno on-the-ground presence is required. In one embodiment, all input datamay be collected via machine data, remote sensing, or crop modeling. Inanother embodiment, the input data may be analyzed, assessed, andcompared to actual outbreaks of disease via machine learning and/orother artificial intelligence techniques to predict which fields, orportions thereof, are most likely to experience a disease outbreak orpropagation. In yet another embodiment, particular agriculturalmanagement techniques (e.g., specific fungicides) may be analyzed,assessed, and/or compared to other treatment options (e.g., otherfungicides) or controls (e.g., non-treatment) to determine the efficacyparticular treatment options have in terms of improved crop vigor,yield, or other indicia of effectiveness. In one aspect of thisembodiment, treatment efficacy may be determined using machine data orremote sensing.

Artificial Intelligence Input Modules and Further Outputs

FIG. 3 illustrates a process for the generation of a treatment map 352for a crop field, in accordance with another embodiment of the presentinvention. The process of FIG. 3 receives two inputs: (a) geospatialimage data 301 (e.g., aerial images of the crop field), and (b)microclimate data 330 pertaining to the crop field. The process feedsthe received geospatial image data 301 to two machine vision modulesexecutable by a hardware processor: a first machine vision module (MVModule 1) 310 to generate a crop vigor map 311, and a second machinevision module (MV Module 2) to estimate crop field parameters 301 (e.g.,canopy closure). The machine vision algorithms used in some embodimentsto determine the crop vigor map and the crop field parameters aredescribed in further detail in FIG. 7.

The process also extracts microclimate parameters or a microclimate map331 from the received microclimate data 330, where the microclimate mapcomprises at least one microclimate parameter mapped to a geolocation ofthe crop field. In FIG. 3, the microclimate parameters 331 arerepresented as square dots on the crop field image.

In addition, the process has an optional third input comprising cropfield parameters 340 other than the parameters 321 estimated by thesecond machine vision module. In one embodiment, the other crop fieldparameters 340 are machine data received from agricultural equipment orother equipment. The crop field parameters estimated by the secondmachine vision algorithm 320 or received in the step 340 may include thecrop row spacing at one or more geolocations of the crop field, theirrigation status, the degree of canopy closure, or the crop stage, eachestimated or received for one or more geolocations of the crop field. Inone embodiment, the irrigation status may be estimated using geospatialimage data and/or received from machine data from one or more irrigationequipment. In one embodiment, the degree of canopy closure may beestimated from the geospatial image data using the same machine visionalgorithm used in step 114. In one embodiment, the crop stage may bedetermined from physical observation and/or one or more crop models.

At step 350, the process applies a risk model to predict pestsusceptibility based on the determined crop vigor map 311, the estimatedcrop field parameters 321, the microclimate parameters 331, and theother (received) crop field parameters 340, where the predicted pestsusceptibility comprises at least one pest susceptibility index of thecrop field. Finally, the process outputs the predicted pestsusceptibility at step 351.

In some embodiments, the process generates a treatment plan (not shownin FIGS. 1A and 1B) based on the generated pest susceptibility map, asshown in FIG. 3. The generated treatment plan comprises a recommendedagricultural management technique (e.g., application of one or moreagricultural chemicals) to prevent the outbreak or control thepropagation of one or more crop pests.

In some embodiments, the process further receives price information fora crop growing in the crop field, cost information for one or moreagricultural management techniques, and an anticipated efficacy for theagricultural management techniques. The process then generates ananticipated return on investment (ROI) based on the price and costinformation and the anticipated efficacy, as discussed below.

FIG. 4 shows another illustrative flowchart of a process for pestsusceptibility determination, in accordance with yet another embodimentof the present invention. The process begins at step 401, where theprocess receives geospatial image data and microclimate data for one ormore crop fields. At step 402, the process determines, utilizing ahardware processor, a crop vigor map for the crop fields from thegeospatial image data. At step 403, the process determines, utilizingthe hardware processor, a row-spacing of a crop fields from thegeospatial image data or from machine data. At step 404, the processdetermines, utilizing the hardware processor, an irrigation status ofthe crop fields from the geospatial image data or from machine data. Atstep 405, the process estimates, utilizing the hardware processor, acrop stage of the crop fields based upon physical observation and/or acrop stage model. At step 406, the process determines, utilizing thehardware processor, whether the crop fields, and what portions of thecrop fields, have reached canopy closure from the geospatial image data.At step 407, the process applies, utilizing the hardware processor, arisk model to determine what portions of the crop fields are amenable topest or disease outbreak and/or propagation, and thus warrant eitherfull or partial treatment. In some embodiments, the risk model utilizesone or more of the determined row-spacing, the irrigation status, thecrop stage, and the portion(s) of canopy closure as inputs. In someembodiments, the system further takes into account how the fields, orportions thereof, are located spatially and/or topographically, toprovide an optimal or near-optimal treatment plan to prevent theoutbreak, or control the propagation, of a crop pest.

Automatically Collecting Crop Field Parameters

In some embodiments, agronomic input data (row spacing,irrigated/non-irrigated determination, crop stage determination, canopyclosure determination) may be determined automatically as illustrated inFIGS. 3 and 4 or calculated as follows. In one embodiment, row spacingis determining from either machine data (e.g., from the tractor orplanter records) or from remote geospatial image data (e.g., aerial,satellite, drone, or in-field sensing data and/or imagery). The tractoror planter records may be obtained via a network, in cases where suchdata is available. Such data may also be downloaded by hard wireconnection from the tractor or planter. Machine data may be obtainedfrom third parties, such as agricultural equipment dealers, agriculturalretailers or consultants, or the growers themselves. In cases wheremachine data is not available or not easily accessed, geospatial imagedata and analytics may be used to estimate row spacing, as describedbelow.

In one embodiment, a determination may be made whether the crop field isirrigated or non-irrigated from either machine data or geospatial imagedata. In one embodiment, geospatial image data is utilized to determinefield shape or other indicators of irrigation, including either thermalor other imagery which may detect irrigation application. Wavelengthscapable of detecting whether a field is irrigated includelong-wavelength infrared (8,000-15,000 nanometers (nm), 20-37 THz),short-wavelength infrared (1,400-3,000 nm, 100-214 THz), and C-Bandradio frequencies (4-8 GHz, as used in Synthetic Aperture Radar).

In some embodiments, an estimation may be made of crop stage through theuse of physical observation of the field in question or one or more cropstage models. In one embodiment, the crop stage models may estimate cropstage using a combination of data sources including (but not limitedto), a measure or estimate of planting or emergence date, a measure orestimate of “thermal time” (i.e., accumulated number hours above atemperature threshold), a measure or estimate of “photoperiod” (i.e.,the length of daylight) at critical intervals, etc.

In one embodiment, the Iowa State Soybean Development Calculator,available athttp://agron.iastate.edu/CroppingSystemsTools/soybean-decisions.html(retrieved in May 2020) and hereby incorporated by reference herein asif fully set forth herein, may be utilized.

In some embodiments, geospatial image data and machine vision algorithmsmay be used to determine quantitatively a degree of canopy closure ofthe crop field. Canopy closure may create in-field conditions needed forthe outbreak or propagation of a pest in the particular field at a timethat other conditions in a general region of the field are suitable tothe pest. If the entire field has not reached the requisite in-fieldconditions (e.g., reproductive stage and near-full canopy closure) atthe relevant time, identifying those areas of the field that havereached the requisite conditions conducive to pest outbreak andpropagation may allow for targeted implementation of agriculturalmanagement techniques (e.g., applications of specific chemicals such asfungicides) to prevent the outbreak, or control the propagation, of croppest(s).

In some embodiments, the potential for pest outbreak or propagation isestimated in particular fields or portions thereof based upon specificdata parameters, including (but not limited to), the crop row spacing,the irrigation status, the degree of canopy closure, and/or the cropstage.

Geospatial Image Data and Analytics to Determine Crop Field Parameters

In one embodiment, geospatial image data and machine vision algorithmsmay be used to determine crop field parameters. For example, machinevision algorithms may be used to measure row spacing and determinequantitatively the degree of canopy closure of the crop field. In oneembodiment, the geospatial image data collection is performed aeriallyusing an aircraft. The same aircraft may also be used to determine bothrow spacing and canopy closure. In another embodiment, the geospatialimage data collection is performed via satellite. In yet anotherembodiment, the geospatial image data collection is performed using adrone. In some embodiments, imagery data (satellite, aerial, and/ordrone) may also be obtained from third parties, such as imagery vendorsserving the agricultural sector.

In some embodiments, in order to measure the distance between the rowsto determine row spacing as well as canopy closure, imagery (eithersatellite, aerial, or drone) must be geo-referenced or geo-rectified andmay need to be ortho-mosaiced.

Using remote geospatial image data and computer vision algorithms allowsmeasurement and quantification of crop field parameters, such as rowspacing and canopy closure, across the entire field, and thus provides amore accurate measure across an entire field or specific portions of thefield. One benefit of the present invention is measuring quantitativelythe crop field parameters across the whole field and providing pestsusceptibility or treatment maps, all being performed remotely,automatically, and accurately.

In some embodiments, the measurement of crop field parameters utilizesone or more machine vision algorithms. Various machine vision algorithmsare within the scope of the present invention. Illustrative machinevision algorithms utilizing a convolutional neural network (CNN)architecture are described below in reference to FIG. 7 below.

Risk Model Embodiments

In one embodiment, a risk model (108, 130, 220, 350) is utilized todetermine which fields, or portions thereof, exhibit conditions amenableto pest outbreak or propagation. Various models are within the scope ofthe present invention.

Existing risk models have been shown to have high accuracy. However,none are known to reach 100% accuracy (i.e., correct pest predictions).For example, 82% to 91% accuracy is observed in R1 to R3 soybean stagefor the weather-based model described in the following publication,which is incorporated by reference as if fully set forth herein:Weather-Based Models for Assessing the Risk of Sclerotinia sclerotiorumApothecial Presence in Soybean (Glycine max) Fields, November 2017,available athttps://apsjournals.apsnet.org/doi/10.1094/PDIS-04-17-0504-RE.Similarly, 65.2% to 78.8% accuracy is observed for the apothecial modeldescribed in the following publication, which is incorporated byreference as if fully set forth herein: Validating Sclerotiniasclerotiorum Apothecial Models to Predict Sclerotinia Stem Rot inSoybean (Glycine max) Fields, October 2018, availablehttps://apsjournals.apsnet.org/doi/10.1094/PDIS-02-18-0245-RE, dependingon which disease incidence thresholds were used. In one embodiment, arisk model may combine existing weather-based models andcrop-parameter-based models to map the risk of a white mold outbreak atdifferent locations of a soybean crop field with high accuracy.

As mentioned above, the risk model may utilize a machine learning (ML)algorithm, such as a random forest, in some embodiments. Various machinelearning algorithms are within the scope of the present invention, andillustrative machine learning algorithms for implementing the risk modelare described below. Illustrative machine learning algorithms utilizinga random forest architecture are described in reference to FIG. 8 below.

In one embodiment, an output (231, 407) of the risk model is adetermination of what portion(s) of the crop field in which to implementagricultural management techniques (e.g., applications of specificchemicals such as fungicides) to prevent the outbreak, or control thepropagation, of crop pests(s). The output may take the form of atreatment map (231, 352). In instances where the entire field may not besusceptible to an outbreak (e.g., because the entire field is not atcanopy closure in the case of white mold), the present invention maydevelop sub-field scale maps to identify areas of the field susceptibleto pest outbreak and/or propagation. This type of output is furtherillustrated in FIGS. 13A and 13B.

In one embodiment, the risk model also receives pricing and costinformation on the costs and efficacy of applying one or moreagricultural management technique(s) in order to facilitate cost-benefitanalysis on the techniques. The pricing and cost information may bereceived from one or more third-party data sources or may be modeledusing one or more correlation or regression models.

In one embodiment, once a given crop field is identified as a likelycandidate for a pest outbreak, the system may compare a likely efficacyof various treatment options based upon published data, data frommachines, data from manufacturers or retailers of treatment techniques,agricultural consultants, growers, third party vendors, or CERESinternal data. In one embodiment, the efficacy determinations areperformed through one or more machine learning (ML) algorithms, such asa random forest algorithm, described in relation to FIG. 8. In oneembodiment, the ML algorithm predicts which treatment is likely to bemost effective given various parameters, including the microclimatedata, soil data, the crop stage, crop variety, and so forth. As aresult, the likely benefits of various treatment options are thenpredicted by the ML algorithm in a quantitative manner (e.g., in termsof expected increases in crop yield). The yield benefits are thenconverted into economic estimates by, in one illustrative aspect,multiplying the yield benefits by the price of the particular crop inquestion. The economic benefits are then compared to the cost of thetreatment option(s) to predict a likely return on investment (ROI) ofvarious treatment options.

Accordingly, and in accordance to one embodiment, the risk modelimplementation further comprises program code to receive priceinformation for a crop growing in the crop field, a cost information forone or more agricultural management techniques, and an anticipatedefficacy for the agricultural management techniques, and generate ananticipated return on investment (ROI) based on the price and costinformation and the anticipated efficacy.

Implementation Using Computer Program Products, Methods, and ComputingEntities

The present invention may be implemented in a combination of hardwareand/or software. An illustrative hardware and software operationalenvironment for implementing one embodiment of the present invention isnow described.

Embodiments of the present disclosure may be implemented in variousways, including as computer program products that comprise articles ofmanufacture. A computer program product may include a non-transitorycomputer-readable storage medium storing applications, programs, programmodules, scripts, source code, program code, object code, byte code,compiled code, interpreted code, machine code, executable instructions,and/or the like (also referred to herein as executable instructions,instructions for execution, computer program products, program code,and/or similar terms used herein interchangeably). Such non-transitorycomputer-readable storage media include all computer-readable media(including volatile and non-volatile media).

In one embodiment, a non-volatile computer-readable storage medium mayinclude a floppy disk, flexible disk, hard disk, solid-state storage(SSS) (e.g., a solid state drive (SSD), solid state card (SSC), solidstate module (SSM), enterprise flash drive, magnetic tape, or any othernon-transitory magnetic medium, and/or the like. A non-volatilecomputer-readable storage medium may also include a punch card, papertape, optical mark sheet (or any other physical medium with patterns ofholes or other optically recognizable indicia), compact disc read onlymemory (CD-ROM), compact disc-rewritable (CD-RW), digital versatile disc(DVD), Blu-ray disc (BD), any other non-transitory optical medium,and/or the like. Such a non-volatile computer-readable storage mediummay also include read-only memory (ROM), programmable read-only memory(PROM), erasable programmable read-only memory (EPROM), electricallyerasable programmable read-only memory (EEPROM), flash memory (e.g.,Serial, NAND, NOR, and/or the like), multimedia memory cards (MMC),secure digital (SD) memory cards, SmartMedia cards, CompactFlash (CF)cards, Memory Sticks, and/or the like. Further, a non-volatilecomputer-readable storage medium may also include conductive-bridgingrandom access memory (CBRAM), phase-change random access memory (PRAM),ferroelectric random-access memory (FeRAIVI), non-volatile random-accessmemory (NVRAM), magnetoresistive random-access memory (MRAM), resistiverandom-access memory (RRAM), Silicon-Oxide-Nitride-Oxide-Silicon memory(SONOS), floating junction gate random access memory (FJG RAM),Millipede memory, racetrack memory, and/or the like.

In one embodiment, a volatile computer-readable storage medium mayinclude random access memory (RAM), dynamic random access memory (DRAM),static random access memory (SRAM), fast page mode dynamic random accessmemory (FPM DRAM), extended data-out dynamic random access memory (EDODRAM), synchronous dynamic random access memory (SDRAM), double datarate synchronous dynamic random access memory (DDR SDRAM), double datarate type two synchronous dynamic random access memory (DDR2 SDRAM),double data rate type three synchronous dynamic random access memory(DDR3 SDRAM), Rambus dynamic random access memory (RDRAM), TwinTransistor RAM (TTRAM), Thyristor RAM (T-RAM), Zero-capacitor (Z-RAM),Rambus in-line memory module (RIMM), dual in-line memory module (DIMM),single in-line memory module (SIMM), video random access memory (VRAM),cache memory (including various levels), flash memory, register memory,and/or the like. It will be appreciated that where embodiments aredescribed to use a computer-readable storage medium, other types ofcomputer-readable storage media may be substituted for or used inaddition to the computer-readable storage media described above.

As should be appreciated, various embodiments of the present disclosuremay also be implemented as methods, apparatus, systems, computingdevices, computing entities, and/or the like. As such, embodiments ofthe present disclosure may take the form of an apparatus, system,computing device, computing entity, and/or the like executinginstructions stored on a computer-readable storage medium to performcertain steps or operations. Thus, embodiments of the present disclosuremay also take the form of an entirely hardware embodiment, an entirelycomputer program product embodiment, and/or an embodiment that comprisescombination of computer program products and hardware performing certainsteps or operations.

Embodiments of the present disclosure are described with reference toblock diagrams and flowchart illustrations. Thus, it should beunderstood that each block of the block diagrams and flowchartillustrations may be implemented in the form of a computer programproduct, an entirely hardware embodiment, a combination of hardware andcomputer program products, and/or apparatus, systems, computing devices,computing entities, and/or the like carrying out instructions,operations, steps, and similar words used interchangeably (e.g., theexecutable instructions, instructions for execution, program code,and/or the like) on a computer-readable storage medium for execution.For example, retrieval, loading, and execution of code may be performedsequentially such that one instruction is retrieved, loaded, andexecuted at a time. In some exemplary embodiments, retrieval, loading,and/or execution may be performed in parallel such that multipleinstructions are retrieved, loaded, and/or executed together. Thus, suchembodiments can produce specifically-configured machines performing thesteps or operations specified in the block diagrams and flowchartillustrations. Accordingly, the block diagrams and flowchartillustrations support various combinations of embodiments for performingthe specified instructions, operations, or steps.

Exemplary System Architecture

An exemplary embodiment of the present disclosure may include one ormore servers (management computing entities), one or more networks, andone or more clients (user computing entities). Each of these components,entities, devices, systems, and similar words used hereininterchangeably may be in direct or indirect communication with, forexample, one another over the same or different wired or wirelessnetworks. Additionally, while FIGS. 5-6 illustrate the various systementities as separate, standalone entities, the various embodiments arenot limited to this particular architecture.

Exemplary Management Computing Entity

FIG. 5 provides a schematic of a server (management computing entity)501 according to one embodiment of the present disclosure. In general,the terms computing entity, computer, entity, device, system, and/orsimilar words used herein interchangeably may refer to, for example, oneor more computers, computing entities, desktop computers, mobile phones,tablets, phablets, notebooks, laptops, distributed systems, gamingconsoles, watches, glasses, iBeacons, proximity beacons, key fobs, radiofrequency identification (RFID) tags, ear pieces, scanners, televisions,dongles, cameras, wristbands, wearable items/devices, kiosks, inputterminals, servers or server networks, blades, gateways, switches,processing devices, processing entities, set-top boxes, relays, routers,network access points, base stations, the like, and/or any combinationof devices or entities adapted to perform the functions, operations,and/or processes described herein. Such functions, operations, and/orprocesses may include, for example, transmitting, receiving, operatingon, processing, displaying, storing, determining, creating/generating,monitoring, evaluating, comparing, and/or similar terms used hereininterchangeably. In one embodiment, these functions, operations, and/orprocesses can be performed on data, content, information, and/or similarterms used herein interchangeably.

As indicated, in one embodiment, the management computing entity 501 mayalso include one or more communications interfaces 520 for communicatingwith various computing entities, such as by communicating data, content,information, and/or similar terms used herein interchangeably that canbe transmitted, received, operated on, processed, displayed, stored,and/or the like. For instance, the management computing entity 501 maycommunicate with user computing entities 601 and/or a variety of othercomputing entities providing geospatial image data 301, microclimatedata 330, or other crop field parameters 340.

As shown in FIG. 5, in one embodiment, the management computing entity501 may include or be in communication with one or more processingelements 505 (also referred to as processors, processing circuitry,and/or similar terms used herein interchangeably) that communicate withother elements within the management computing entity 501 via a bus, forexample. As will be understood, the processing element 505 may beembodied in a number of different ways. For example, the processingelement 505 may be embodied as one or more complex programmable logicdevices (CPLDs), microprocessors, multi-core processors, coprocessingentities, application-specific instruction-set processors (ASIPs),microcontrollers, and/or controllers. Further, the processing element505 may be embodied as one or more other processing devices orcircuitry. The term circuitry may refer to an entirely hardwareembodiment or a combination of hardware and computer program products.Thus, the processing element 505 may be embodied as integrated circuits,application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), programmable logic arrays (PLAs), hardwareaccelerators, other circuitry, and/or the like. As will therefore beunderstood, the processing element 505 may be configured for aparticular use or configured to execute instructions stored in volatileor non-volatile media or otherwise accessible to the processing element505. As such, whether configured by hardware or computer programproducts, or by a combination thereof, the processing element 505 may becapable of performing steps or operations according to embodiments ofthe present disclosure when configured accordingly.

In one embodiment, the management computing entity 501 may furtherinclude or be in communication with non-volatile media (also referred toas non-volatile storage, memory, memory storage, memory circuitry and/orsimilar terms used herein interchangeably). In one embodiment, thenon-volatile storage or memory may include one or more non-volatilestorage or memory media 510, including but not limited to hard disks,ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards, MemorySticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipedememory, racetrack memory, and/or the like. As will be recognized, thenon-volatile storage or memory media may store databases, databaseinstances, database management systems, data, applications, programs,program modules, scripts, source code, object code, byte code, compiledcode, interpreted code, machine code, executable instructions, and/orthe like. The term database, database instance, database managementsystem, and/or similar terms used herein interchangeably may refer to acollection of records or data that is stored in a computer-readablestorage medium using one or more database models, such as a hierarchicaldatabase model, network model, relational model, entity-relationshipmodel, object model, document model, semantic model, graph model, and/orthe like.

In one embodiment, the management computing entity 501 may furtherinclude or be in communication with volatile media (also referred to asvolatile storage, memory, memory storage, memory circuitry and/orsimilar terms used herein interchangeably). In one embodiment, thevolatile storage or memory may also include one or more volatile storageor memory media 515, including but not limited to RAM, DRAM, SRAM, FPMDRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, TTRAM,T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory,and/or the like. As will be recognized, the volatile storage or memorymedia may be used to store at least portions of the databases, databaseinstances, database management systems, data, applications, programs,program modules, scripts, source code, object code, byte code, compiledcode, interpreted code, machine code, executable instructions, and/orthe like being executed by, for example, the processing element 505.Thus, the databases, database instances, database management systems,data, applications, programs, program modules, scripts, source code,object code, byte code, compiled code, interpreted code, machine code,executable instructions, and/or the like may be used to control certainaspects of the operation of the management computing entity 501 with theassistance of the processing element 505 and operating system.

As indicated, in one embodiment, the management computing entity 501 mayalso include one or more communications interfaces 520 for communicatingwith various computing entities, such as by communicating data, content,information, and/or similar terms used herein interchangeably that canbe transmitted, received, operated on, processed, displayed, stored,and/or the like. Such communication may be executed using a wired datatransmission protocol, such as fiber distributed data interface (FDDI),digital subscriber line (DSL), Ethernet, asynchronous transfer mode(ATM), frame relay, data over cable service interface specification (DOCSIS), or any other wired transmission protocol. Similarly, themanagement computing entity 501 may be configured to communicate viawireless external communication networks using any of a variety ofprotocols, such as general packet radio service (GPRS), Universal MobileTelecommunications System (UMTS), Code Division Multiple Access 2000(CDMA2000), CDMA2000 1× (1×RTT), Wideband Code Division Multiple Access(WCDMA), Time Division-Synchronous Code Division Multiple Access(TD-SCDMA), Long Term Evolution (LTE), Evolved Universal TerrestrialRadio Access Network (E-UTRAN), Evolution-Data Optimized (EVDO), HighSpeed Packet Access (HSPA), High-Speed Downlink Packet Access (HSDPA),IEEE 802.11 (Wi-Fi), Wi-Fi Direct, 802.16 (WiMAX), ultra-wideband (UWB),infrared (IR) protocols, near field communication (NFC) protocols,Wibree, Bluetooth protocols, wireless universal serial bus (USB)protocols, and/or any other wireless protocol.

Although not shown, the management computing entity 501 may include orbe in communication with one or more input elements, such as a keyboardinput, a mouse input, a touch screen/display input, motion input,movement input, audio input, pointing device input, joystick input,keypad input, and/or the like. The management computing entity 501 mayalso include or be in communication with one or more output elements(not shown), such as audio output, video output, screen/display output,motion output, movement output, and/or the like.

As will be appreciated, one or more of the components of the managementcomputing entity 501 may be located remotely from other managementcomputing entity 501 components, such as in a distributed system.Furthermore, one or more of the components may be combined andadditional components performing functions described herein may beincluded in the management computing entity 501. Thus, the managementcomputing entity 501 can be adapted to accommodate a variety of needsand circumstances. As will be recognized, these architectures anddescriptions are provided for exemplary purposes only and are notlimiting to the various embodiments.

Exemplary User Computing Entity

A user may be an individual, a company, an organization, an entity, adepartment within an organization, a representative of an organizationand/or person, and/or the like. FIG. 6 provides an illustrativeschematic representative of a client (user computing entity) 601 thatcan be used in conjunction with embodiments of the present disclosure.In general, the terms device, system, computing entity, entity, and/orsimilar words used herein interchangeably may refer to, for example, oneor more computers, computing entities, desktops, mobile phones, tablets,phablets, notebooks, laptops, distributed systems, gaming consoles,watches, glasses, key fobs, radio frequency identification (RFID) tags,ear pieces, scanners, cameras, wristbands, kiosks, input terminals,servers or server networks, blades, gateways, switches, processingdevices, processing entities, set-top boxes, relays, routers, networkaccess points, base stations, the like, and/or any combination ofdevices or entities adapted to perform the functions, operations, and/orprocesses described herein. User computing entities 601 can be operatedby various parties. As shown in FIG. 6, the user computing entity 601can include an antenna 612, a transmitter 604 (e.g., radio), a receiver606 (e.g., radio), and a processing element 608 (e.g., CPLDs,microprocessors, multi-core processors, coprocessing entities, ASIPs,microcontrollers, and/or controllers) that provides signals to andreceives signals from the transmitter 604 and receiver 606,respectively.

The signals provided to and received from the transmitter 604 and thereceiver 606, respectively, may include signalling information inaccordance with air interface standards of applicable wireless systems.In this regard, the user computing entity 601 may be capable ofoperating with one or more air interface standards, communicationprotocols, modulation types, and access types. More particularly, theuser computing entity 601 may operate in accordance with any of a numberof wireless communication standards and protocols, such as thosedescribed above with regard to the management computing entity 501. In aparticular embodiment, the user computing entity 601 may operate inaccordance with multiple wireless communication standards and protocols,such as UMTS, CDMA2000, 1×RTT, WCDMA, TD-SCDMA, LTE, E-UTRAN, EVDO,HSPA, HSDPA, Wi-Fi, Wi-Fi Direct, WiMAX, UWB, IR, NFC, Bluetooth, USB,and/or the like. Similarly, the user computing entity 601 may operate inaccordance with multiple wired communication standards and protocols,such as those described above with regard to the management computingentity 501 via a network interface 620.

Via these communication standards and protocols, the user computingentity 601 can communicate with various other entities using conceptssuch as Unstructured Supplementary Service Data (USSD), Short MessageService (SMS), Multimedia Messaging Service (MMS), Dual-ToneMulti-Frequency Signalling (DTMF), and/or Subscriber Identity ModuleDialer (SIM dialer). The user computing entity 601 can also downloadchanges, add-ons, and updates, for instance, to its firmware, software(e.g., including executable instructions, applications, programmodules), and operating system.

According to one embodiment, the user computing entity 601 may includelocation determining aspects, devices, modules, functionalities, and/orsimilar words used herein interchangeably. For example, the usercomputing entity 601 may include outdoor positioning aspects, such as alocation module adapted to acquire, for example, latitude, longitude,altitude, geocode, course, direction, heading, speed, universal time(UTC), date, and/or various other information/data. In one embodiment,the location module can acquire data, sometimes known as ephemeris data,by identifying the number of satellites in view and the relativepositions of those satellites. The satellites may be a variety ofdifferent satellites, including Low Earth Orbit (LEO) satellite systems,Department of Defense (DOD) satellite systems, the European UnionGalileo positioning systems, the Chinese Compass navigation systems,Indian Regional Navigational satellite systems, and/or the like.Alternatively, the location information can be determined bytriangulating the user computing entity's 601 position in connectionwith a variety of other systems, including cellular towers, Wi-Fi accesspoints, and/or the like. Similarly, the user computing entity 601 mayinclude indoor positioning aspects, such as a location module adapted toacquire, for example, latitude, longitude, altitude, geocode, course,direction, heading, speed, time, date, and/or various otherinformation/data. Some of the indoor systems may use various position orlocation technologies including RFID tags, indoor beacons ortransmitters, Wi-Fi access points, cellular towers, nearby computingdevices (e.g., smartphones, laptops) and/or the like. For instance, suchtechnologies may include the iBeacons, Gimbal proximity beacons,Bluetooth Low Energy (BLE) transmitters, NFC transmitters, and/or thelike. These indoor positioning aspects can be used in a variety ofsettings to determine the location of someone or something to withininches or centimeters.

The user computing entity 601 may also comprise a user interface (thatcan include a display 616 coupled to a processing element 608) and/or auser input interface (coupled to a processing element 608). For example,the user interface may be a user application, browser, user interface,and/or similar words used herein interchangeably executing on and/oraccessible via the user computing entity 601 to interact with and/orcause display of information from the management computing entity 501,as described herein. The user input interface can comprise any of anumber of devices or interfaces allowing the user computing entity 601to receive data, such as a keypad 618 (hard or soft), a touch display,voice/speech or motion interfaces, or other input device. In embodimentsincluding a keypad 618, the keypad 618 can include (or cause display of)the conventional numeric (0-9) and related keys (#, *), and other keysused for operating the user computing entity 601 and may include a fullset of alphabetic keys or set of keys that may be activated to provide afull set of alphanumeric keys. In addition to providing input, the userinput interface can be used, for example, to activate or deactivatecertain functions, such as screen savers and/or sleep modes.

The user computing entity 601 can also include volatile storage ormemory 622 and/or non-volatile storage or memory 624, which can beembedded and/or may be removable. For example, the non-volatile memorymay be ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards,Memory Sticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM,Millipede memory, racetrack memory, and/or the like. The volatile memorymay be RAM, DRAM, SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2SDRAM, DDR3 SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM,cache memory, register memory, and/or the like. The volatile andnon-volatile storage or memory can store databases, database instances,database management systems, data, applications, programs, programmodules, scripts, source code, object code, byte code, compiled code,interpreted code, machine code, executable instructions, and/or the liketo implement the functions of the user computing entity 601. Asindicated, this may include a user application that is resident on theentity or accessible through a browser or other user interface forcommunicating with the management computing entity 501 and/or variousother computing entities.

In another embodiment, the user computing entity 601 may include one ormore components or functionality that are the same or similar to thoseof the management computing entity 501, as described in greater detailabove. As will be recognized, these architectures and descriptions areprovided for exemplary purposes only and are not limiting to the variousembodiments.

Machine Vision and Machine Learning Modules

The present invention may be implemented using one or more machinevision and machine learning modules implementing one or more algorithmsimplemented in non-transitory storage medium having program code storedthereon, the program code executable by one or more processors, asdescribed above. The following description describes in detail some ofthe illustrative machine vision and machine learning algorithms usefulfor implementing some embodiments of the present invention.

Illustrative Machine Vision Architectures

Various exemplary machine vision algorithms are within the scope of thepresent invention used for generating a crop vigor map 311 and forestimating crop parameters 321 such as crop row spacing, canopy closure,and so forth, from geospatial image data 301.

Some exemplary machine vision algorithms utilize a deep learning network(DLN), for example using a convolutional neural network (CNN). FIG. 7shows an exemplary CNN module that may be utilized for implementingvarious machine vision algorithms described herein. In FIG. 7, one ormore input layers 702 are connected via a multiplicity of hidden layers704 to one or more output layers 706. This neural network architecturemay receive geospatial image data 701 and may be trained to generate 708a crop vigor map, a crop vigor index, or to estimate 708 row spacing,canopy closure, and other machine vision tasks required by the presentinvention. FIG. 7 shows only one illustrative CNN architecture that iswithin the scope of the present invention, and the present invention isnot limited to the use of CNNs. Other machine vision algorithms are alsowithin the scope of the present invention.

Illustrative Machine Learning Architectures

Various exemplary machine learning algorithms may be used within thescope of the present invention as a risk model 350 to determine pestsusceptibility 351 from a crop vigor map 311, microclimate data 330, andother estimated 321 or received 340 crop field parameters.

FIG. 8 shows an illustrative diagram for a machine learning algorithmused to implement the risk model 350, in accordance with one embodimentof the invention. In one embodiment, the machine learning algorithmcomprises a random forest algorithm, one illustrative machine learningalgorithm. Random forest algorithms use a multitude of decision treepredictors, such that each decision tree depends on the values of arandom subset of the training data, which minimizes the chances ofoverfitting to the training data set. In one embodiment, the randomforest algorithm is implementation as described in Leo Breiman, RandomForests, Machine Learning, 45, 5-32, 2001, Kluwer Academic Publishers,Netherlands, available at doi.org/10.1023/A:1010933404324. Random forestis only one illustrative machine learning algorithm that is within thescope of the present invention, and the present invention is not limitedto the use of random forest. Other machine learning algorithms,including but not limited to, nearest neighbor, decision trees, supportvector machines (SVM), Adaboost, Bayesian networks, various neuralnetworks including deep learning networks, evolutionary algorithms, andso forth, are within the scope of the present invention. The input tothe machine learning algorithm is a feature vector 802, comprising theinput data described above (i.e., crop vigor index, microclimate data,etc.). The output of the machine learning algorithm are the predictedpest susceptibility indices 808 for the different portions of the cropfield.

As noted, embodiments of devices and systems (and their variouscomponents) described herein can employ artificial intelligence (AI) tofacilitate automating one or more features described herein. Thecomponents can employ various AI-based schemes for carrying out variousembodiments/examples disclosed herein. To provide for or aid in thenumerous determinations (e.g., determine, ascertain, infer, calculate,predict, prognose, estimate, derive, forecast, detect, compute)described herein, components described herein can examine the entiretyor a subset of the data to which it is granted access and can providefor reasoning about or determine states of the system, environment, etc.from a set of observations as captured via events and/or data.Determinations can be employed to identify a specific context or action,or can generate a probability distribution over states, for example. Thedeterminations can be probabilistic—that is, the computation of aprobability distribution over states of interest based on aconsideration of data and events.

Components disclosed herein can employ various classification(explicitly trained (e.g., via training data) as well as implicitlytrained (e.g., via observing behavior, preferences, historicalinformation, receiving extrinsic information, etc.)) schemes and/orsystems (e.g., support vector machines, neural networks, expert systems,Bayesian belief networks, fuzzy logic, data fusion engines, etc.) inconnection with performing automatic and/or determined action inconnection with the claimed subject matter. Thus, classification schemesand/or systems can be used to automatically learn and perform a numberof functions, actions, and/or determinations.

A classifier may map an input attribute vector, z=(z₁, z₂, . . . ,z_(n)), to a confidence that the input belongs to a class, as byf(z)=confidence(class). Such classification may employ a probabilisticand/or statistical-based analysis (e.g., factoring into the analysisutilities and costs) to determinate an action to be automaticallyperformed. Another example of a classifier that can be employed is asupport vector machine (SVM). The SVM operates by finding ahyper-surface in the space of possible inputs, where the hyper-surfaceattempts to split the triggering criteria from the non-triggeringevents. Intuitively, this makes the classification correct for testingdata that is near, but not identical to training data. Other directedand undirected model classification approaches can be employed,including, e.g., naive Bayes, Bayesian networks, decision trees, neuralnetworks, fuzzy logic models, and/or probabilistic classification modelsproviding different patterns of independence. Classification as usedherein also is inclusive of statistical regression that is utilized todevelop models of priority.

Training the Machine Learning Algorithms

FIG. 9 shows a diagram of an exemplary flow diagram for training themachine learning (ML) algorithms (e.g., the risk model 350), which areutilized in predicting which fields, or portions thereof, exhibitconditions amenable to pest outbreak or propagation, in accordance withexample embodiments of the present invention. The training processbegins at step 902, with data acquisition. At step 904, the acquireddata is pre-processed (known as data preparation). At step 906, themodel is trained using training data 950. At step 908, the model isevaluated and tested, and further refinements to the model are fed backinto step 906. At step 910, the optimal model parameters are selected.At step 912, the model is deployed. New data 952 can now be used by thedeployed model to make predictions.

In various embodiments of the present invention, training may apply tothe machine vision algorithms described in FIG. 7 or to the machinelearning algorithms described in FIG. 8. For the machine visionalgorithms described above (e.g., 310, 320), the input data acquired atstep 902 may comprise geospatial image data of one or more sample cropfields and one or more crop vigor maps for the one or more sample cropfields. For the machine learning algorithms described above (e.g., therisk model 350), the input data acquired at step 902 may comprise cropvigor map of one or more sample crop fields as well as microclimatedata, other crop field parameters (e.g., row spacing), pest outbreakmeasurements, and/or agricultural treatment plans of the one or moresample crop fields.

A starting point for any machine learning method such as used by themachine learning component above is a documented dataset containingmultiple instances of system inputs and correct outcomes (e.g., thetraining data). This data set can be used, using methods known in theart, including but not limited to standardized machine learning methodssuch as parametric classification methods, non-parametric methods,decision tree learning, neural networks, methods combining bothinductive and analytic learning, and modeling approaches such asregression models, to train the machine learning system and to evaluateand optimize the performance of the trained system. The quality of theoutput of the machine learning system depends on (a) the patternparameterization, (b) the learning machine design, and (c) the qualityof the training database. These components can be refined and optimizedusing various methods. For example, the database can be refined byadding datasets for new documented crop fields. The quality of thedatabase can be improved, for example, by populating the database withcases in which the customization was accomplished by one or more expertsin crop pest prediction or treatment (e.g., fungicide) application.Thus, the database will better represent the expert's knowledge. In oneembodiment, the database includes data, for example, of pooragricultural management, which can assist in the evaluation of a trainedsystem.

CERES Platform

FIG. 10 is a schematic diagram illustrating functionalities provided byan extended CERES platform 1000, according to one embodiment of thepresent invention. In some embodiments, the CERES platform 1000 mayreceive geospatial image data (e.g., remote aerial images) 1010,microclimate data 1020, and other crop field parameters (e.g., cropstage, row spacing, etc.) 1030. The received data may be processed bythe CERES system 1050, and stored in the cloud for later use, analysis,playback, and predictions.

The CERES server 1050 may provide crop pest susceptibility predictions1060, real-time analytics 1070, and/or other agricultural managementrecommendations (e.g., crop field treatment) 1080 to a plurality ofend-user devices over the network.

Although the CERES system 1050 as shown in FIG. 10 serves as the corefor CERES platform 1000, in some embodiments, CERES platform 1000 may benetworked among multiple user devices, where a CERES system 1050 may beconnected to multiple user computing devices, each used to analyze inputdata and display treatment (e.g., fungicide) application data, and forproviding analytics. Such analytics data may be stored at the CERESsystem 1050, which in turn may facilitate sharing of such data amongindividual users, or participants of an online agronomic analyticscommunity.

Example Use Cases and Proof-of-Concept of the Present Invention

FIG. 11 shows an exemplary relationship between canopy temperature andcrop vigor index. The measurements are derived from CERES IMAGINGsensors for a specific crop field and show a plotted regression lineillustrating the linear relationship between the two quantities. Thevariable on the x-axis is a crop vigor index similar to NDVI or NDRE,and the y-axis is temperature in degrees Fahrenheit.

The plot of FIG. 11 highlights how diurnal canopy temperatures (solarnoon) are generally warmer where the canopy is open/weak. Most of the˜10,000 points from this soybean field are clustered at a relativelydense canopy with indices ranging from −0.1 to 0.0. FIG. 11 shows thatthis dense canopy is cooler during the day (i.e., less than 70 degreesFahrenheit). These parts of the field are more susceptible to diseasessuch as white mold. The strong correlation between the crop vigor indexon the x-axis and canopy temperature on the y-axis indicates that canopytemperature derived from aerial or in-field sensors can be used todetermine a crop vigor index, as depicted in the processes of FIGS. 1Aand 1B. Moreover, this strong relationship also indicates that a cropvigor index could be an excellent proxy for microclimate parameters inmicroclimates that strongly influence the presence, absence, or severityof pests. This is a useful feature in some embodiments, since acquiringrobust and reliable thermal data at the field scale may be expensive. Insuch microclimates, it is therefore possible to have embodiments thatuse solely geospatial image data inputs.

In order to develop a spatially explicit output map of fine-scale pestsusceptibility 351 that can be used for precise treatment 352, themachine vision module 310 described in some embodiments above todetermine a crop vigor map 311 requires pest susceptibility data as adependent variable. For purposes of calibration and training, such pestsusceptibility data should include GPS coordinates with correspondingpresence, absence, and/or severity of the pest, as discussed in FIGS.12A, 12B, 13A, and 13B.

FIGS. 12A and 12B demonstrate the viability of the invention throughdata collected by CERES IMAGING from the field. FIG. 12A shows canopytemperatures of a sugarbeet field computed by an aerial thermal camera,whereas FIG. 12B shows the Chlorophyll vegetation index map (i.e., acrop vigor map) calculated for the same crop field on the same flight.It is clear that the crop vigor map correlates strongly with canopytemperature: the spots with open/weak canopy on the Chlorophyllvegetation index map of FIG. 12B (darker spots) correspond to warmerareas on the canopy temperature map of FIG. 12A (lighter spots).Similarly, the spots with thick/dense canopy on the crop vigor mapcorrespond to cooler areas on the canopy temperature map.

In some embodiments of the present invention, the risk model 350described above may use one or more varieties of aerial images collectedas the crop is approaching canopy closure to predict pest susceptibility351. In some embodiments, the crop vigor map 331 may be computed using anormalized difference vegetation index (NDVI), a normalized differencered edge (NDRE) index, and/or a modified chlorophyll absorption ratioindex 2 (MCARI2). In other embodiments, the crop vigor map 331 may becomputed from a thermal image using a correlation computed as in FIG.11.

In some embodiments of the present invention, the risk model 350described above may use a thermal image collected as the crop isapproaching canopy closure to predict pest susceptibility 351. For someembodiments, the crop vigor map 331 may be computed from the thermalimage using a correlation computed as in FIG. 11.

FIG. 13A shows an illustrative example of a crop vigor index map (i.e.,crop vigor map) with an overlay of sample measured pest index datapoints, where the pest in this case is white mold. In FIG. 13A, the cropvigor index on the map is darker (i.e., black) for low vigor and lighter(i.e., white) for high crop vigor. The pest index is measured on thefield. The points in the field where measured pest index (e.g., diseaseseverity scale) are higher are indicated by larger white squares,whereas the points where the pest severity is lower or absent areindicated by the smaller white squares. For the case of white mold inthis particular crop, FIG. 13A illustrates that a high measured pestindex occurs where crop vigor index is high. Therefore, white mold ismore likely (high pest index) in denser canopy (high crop vigor index).

In some embodiments, the map and data points of FIG. 13A may be used inthe training and calibration of the risk model 350, as it links thepoints in the field where the pest index (e.g., disease severity scale)is high, low, or absent, with the corresponding crop vigor index.Therefore, data such as FIG. 13A may constitute training data 950 to beused as ground-truth data for training the machine learning module ofFIG. 8, in certain embodiments of the present invention.

FIG. 13B shows the predicted pest susceptibility map for the examplecrop field of FIG. 13A. FIG. 13B therefore illustrates a sample output351 of the risk model 350. The risk model used to generate the pestsusceptibility map of FIG. 13B is a model where an in-season crop vigorindex is used with the locations of white mold in order to demonstrateproof of concept.

In FIG. 13B, the predicted pest susceptibility index is a predictedseverity measure where 0 corresponds to “no disease” and 100 isconsidered “severe disease”. With a functional resolution of 10 feet by10 feet, the map areas showing a severe pest susceptibility of 100indicate areas where white mold is likely to occur on every plant,whereas the map areas showing a low pest susceptibility of 0 indicateareas where no white mold is expected to occur on any plant.

Pest susceptibility predicted at one point in time depends on thesubsequent macroclimate conditions seen by the crop field. In general,if the broader macroclimate conditions are suitable, the white areaswould have 100% probability of disease occurrence. On the other hand, ifthe broader macroclimate conditions are less suitable, the white areaswould still have a high (around 70%) probability of disease. Conversely,the black areas would have 0% probability of disease, in many cases,regardless of the broader macroclimate conditions. This illustrates thataccurate pest susceptibility maps such as the one in FIG. 13B havecrucial implications on treatment management, planning, and efficiency.

It is important to note that the effect of a shift (e.g., increase) in acrop field parameter such as canopy closure, or an indicator such ascrop vigor index, on any given pest, may depend on the pest. Forexample, an increase in crop vigor is likely to lead to an increasedsusceptibility to white mold, as demonstrated in the maps of FIGS. 13Aand 13B. However, the same increase in crop vigor may lead to reducedsusceptibility to weed and other pests requiring sun exposure.

As mentioned above, the various embodiments of the present inventionapply to any pest, including crop diseases, insects, weeds, and plantpathogens.

Illustrative Sources for Geospatial Image Data and Microclimate Data

The geospatial image data (101, 112, 205, 301, 401) and microclimatedata (104, 122, 205, 330, 401) used as inputs to the processes describedherein may be received from remote sensors located on aircraft. Theaircraft may be manned or unmanned with the unmanned aircraft controlledby a ground-based operator or flying autonomously along a programmedflight path. The data may be mosaicked into a single map or image in anysuitable manner, where the map provides a pixel value for a plurality ofgeographically referenced locations in the field. The geographicallyreferenced locations may be referenced relative to any suitablereference frame, including global coordinates, or to a field-specificlocal reference frame or plant identifier. The term “image” or “map”, asused herein, does not necessarily require or imply a 2-dimensionalrepresentation and refers simply to data having a referenced positionand a value associated with that position, thereby containing theinformation of a two-dimensional map without requiring an actualtwo-dimensional representation.

Aerial imagery may be obtained using one or more cameras mounted on amanned or unmanned aerial vehicle, preferably over a short period oftime near mid-day, although any suitable method may be used withoutdeparting from various aspects of the invention. The images should beacquired over as short a period as possible so that environmentalconditions do not significantly change between the beginning and end ofa data-acquisition flight. To minimize flight time while coveringtypical agricultural areas, the flight altitude should be at least 200meters. For accurately extracting aerial measurements at the locationsof ground samples, individual images are mosaicked, and the mosaic isgeo-registered and orthorectified.

In other embodiments, images are taken from satellites, and a similarprocess is followed for image processing as for aerial images.

In another aspect, the microclimate parameters may be collected by thesame aircraft taking the thermal images. In some embodiments, theaircraft may measure the microclimate parameter(s) during a pass overthe field at a lower elevation than when taking the thermal image(s)that may be above 200 meters in altitude, and on the same day as thethermal images were collected. In other embodiments, the microclimateparameters are estimated using models or “synthetic sensors” derivedfrom broader microclimate or weather models.

In some embodiments, ground-based microclimate measurements may be usedwithout departing from the scope of the present invention. Microclimateparameters may also be obtained from third parties, such as third-partyweather stations, whether ground-based or remote.

Treatment Plan Implementation

In some embodiments, the sprayers used in precision agriculture mayincorporate engineering that allows growers to execute the process at afine scale (i.e., individual nozzle control for on/off sprayapplications at a 20 inch spacing). As an example, flow control may beperformed at the individual nozzle level, which would allow theapplication of different fungicide/spray rates at a 20 inch spacing.This process would take into account the spatial variability of canopyvigor in a closed canopy. In one embodiment, the treatment plangenerates a control protocol for these spray nozzles. Therefore, heavyrates of fungicide/spray would generate a high ROI where canopy isclosed and the most vigorous. And fungicide/spray rates could bedecreased substantially (but not eliminated or turned off) in otherparts of the field where canopy is closed, but far less vigorous, savingmoney for growers.

Additional Implementation Details

Although an example processing system has been described above,implementations of the subject matter and the functional operationsdescribed herein can be implemented in other types of digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them.

Embodiments of the subject matter and the operations described hereincan be implemented in digital electronic circuitry, or in computersoftware, firmware, or hardware, including the structures disclosed inthis specification and their structural equivalents, or in combinationsof one or more of them. Embodiments of the subject matter describedherein can be implemented as one or more computer programs, i.e., one ormore modules of computer program instructions, encoded on computerstorage medium for execution by, or to control the operation of,information/data processing apparatus. Alternatively, or in addition,the program instructions can be encoded on an artificially-generatedpropagated signal, e.g., a machine-generated electrical, optical, orelectromagnetic signal, which is generated to encode information/datafor transmission to suitable receiver apparatus for execution by aninformation/data processing apparatus. A computer storage medium can be,or be included in, a computer-readable storage device, acomputer-readable storage substrate, a random or serial access memoryarray or device, or a combination of one or more of them. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium can be a source or destination of computer programinstructions encoded in an artificially-generated propagated signal. Thecomputer storage medium can also be, or be included in, one or moreseparate physical components or media (e.g., multiple CDs, disks, orother storage devices).

The operations described herein can be implemented as operationsperformed by an information/data processing apparatus oninformation/data stored on one or more computer-readable storage devicesor received from other sources.

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations, of the foregoing. The apparatus can includespecial purpose logic circuitry, e.g., an FPGA (field programmable gatearray) or an ASIC (application-specific integrated circuit). Theapparatus can also include, in addition to hardware, code that createsan execution environment for the computer program in question, e.g.,code that constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, a cross-platform runtimeenvironment, a virtual machine, or a combination of one or more of them.The apparatus and execution environment can realize various differentcomputing model infrastructures, such as web services, distributedcomputing, and grid computing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor information/data (e.g., one or more scripts stored in a markuplanguage document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub-programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described herein can be performed by oneor more programmable processors executing one or more computer programsto perform actions by operating on input information/data and generatingoutput. Processors suitable for the execution of a computer programinclude, by way of example, both general and special purposemicroprocessors, and any one or more processors of any kind of digitalcomputer. Generally, a processor will receive instructions andinformation/data from a read-only memory or a random-access memory orboth. The essential elements of a computer are a processor forperforming actions in accordance with instructions and one or morememory devices for storing instructions and data. Generally, a computerwill also include, or be operatively coupled to receive information/datafrom or transfer information/data to, or both, one or more mass storagedevices for storing data, e.g., magnetic, magneto-optical disks, oroptical disks. However, a computer need not have such devices. Devicessuitable for storing computer program instructions and information/datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subjectmatter described herein can be implemented on a computer having adisplay device, e.g., a CRT (cathode ray tube) or LCD (liquid crystaldisplay) monitor, for displaying information/data to the user and akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input. In addition, a computer can interact with a user bysending documents to and receiving documents from a device that is usedby the user; for example, by sending web pages to a web browser on auser's client device in response to requests received from the webbrowser.

Embodiments of the subject matter described herein can be implemented ina computing system that includes a back-end component, e.g., as aninformation/data server, or that includes a middleware component, e.g.,an application server, or that includes a front-end component, e.g., aclient computer having a graphical user interface or a web browserthrough which a user can interact with an implementation of the subjectmatter described herein, or any combination of one or more suchback-end, middleware, or front-end components. The components of thesystem can be interconnected by any form or medium of digitalinformation/data communication, e.g., a communication network. Examplesof communication networks include a local area network (“LAN”) and awide area network (“WAN”), an inter-network (e.g., the Internet), andpeer-to-peer networks (e.g., ad hoc peer-to-peer networks).

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someembodiments, a server transmits information/data (e.g., an HTML page) toa client device (e.g., for purposes of displaying information/data toand receiving user input from a user interacting with the clientdevice). Information/data generated at the client device (e.g., a resultof the user interaction) can be received from the client device at theserver.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyembodiment or of what may be claimed, but rather as descriptions offeatures specific to particular embodiments. Certain features that aredescribed herein in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable sub-combination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

Thus, particular embodiments of the subject matter have been described.Other embodiments are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous.

In some embodiments of the present invention, the entire system can beimplemented and offered to the end-users and operators over theInternet, in a so-called cloud implementation. No local installation ofsoftware or hardware would be needed, and the end-users and operatorswould be allowed access to the systems of the present invention directlyover the Internet, using either a web browser or similar software on aclient, which client could be a desktop, laptop, mobile device, and soon. This eliminates any need for custom software installation on theclient side and increases the flexibility of delivery of the service(software-as-a-service), and increases user satisfaction and ease ofuse. Various business models, revenue models, and delivery mechanismsfor the present invention are envisioned, and are all to be consideredwithin the scope of the present invention.

In general, the method executed to implement the embodiments of theinvention, may be implemented as part of an operating system or aspecific application, component, program, object, module or sequence ofinstructions referred to as “computer program(s)” or “computer code(s).”The computer programs typically comprise one or more instructions set atvarious times in various memory and storage devices in a computer, andthat, when read and executed by one or more processors in a computer,cause the computer to perform operations necessary to execute elementsinvolving the various aspects of the invention. Moreover, while theinvention has been described in the context of fully functioningcomputers and computer systems, those skilled in the art will appreciatethat the various embodiments of the invention are capable of beingdistributed as a program product in a variety of forms, and that theinvention applies equally regardless of the particular type of machineor computer-readable media used to actually effect the distribution.Examples of computer-readable media include but are not limited torecordable type media such as volatile and non-volatile memory devices,floppy and other removable disks, hard disk drives, optical disks, whichinclude Compact Disk Read-Only Memory (CD ROMS), Digital Versatile Disks(DVDs), etc., as well as digital and analog communication media.

One of ordinary skill in the art knows that the use cases, structures,schematics, and flow diagrams may be performed in other orders orcombinations, but the inventive concept of the present invention remainswithout departing from the broader scope of the invention. Everyembodiment may be unique, and methods/steps may be either shortened orlengthened, overlapped with the other activities, postponed, delayed,and continued after a time gap, such that every user in a client-serverenvironment is accommodated to practice the methods of the presentinvention.

CONCLUSIONS

Many modifications and other embodiments of the disclosure set forthherein will come to mind to one skilled in the art to which theseembodiments pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the embodiments are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

Although the present invention has been described with reference tospecific exemplary embodiments, it will be evident that the variousmodification and changes can be made to these embodiments withoutdeparting from the broader scope of the invention. Accordingly, thespecification and drawings are to be regarded in an illustrative senserather than in a restrictive sense. It will also be apparent to theskilled artisan that the embodiments described above are specificexamples of a single broader invention which may have greater scope thanany of the singular descriptions taught. There may be many alterationsmade in the descriptions without departing from the scope of the presentinvention as defined by the appended claims.

What is claimed is:
 1. A system comprising a hardware processor and a non-transitory storage medium for storing program code, the program code executable by the hardware processor to execute a process for predicting a pest susceptibility, the program code when executed by the hardware processor causing the hardware processor to execute steps to: receive geospatial image data of a crop field from one or more sensors, wherein the geospatial image data of the crop field is geocoded by longitude and latitude coordinates; receive microclimate data of the crop field, wherein the microclimate data comprises locally variable environmental conditions in one or more portions of the crop field; determine a crop vigor map for the crop field from the geospatial image data, wherein the crop vigor map comprises a crop vigor index on the one or more portions of the crop field, and wherein the crop vigor index is a numerical index based on a foliage volume, density, layout, and/or health status of the crop in the crop field; and generate a pest susceptibility map utilizing a risk model based on the crop vigor map and the microclimate data, wherein the pest susceptibility map comprises a pest susceptibility index geocoded by latitude and longitude coordinates, and wherein the pest susceptibility index is a measure of a susceptibility of a crop in the crop field to one or more crop pests.
 2. The system of claim 1, wherein the determining the crop vigor map for the crop field comprises calculating the crop vigor index from one or more crop vigor indicator equations using the geospatial image data as input.
 3. The system of claim 1, wherein the determining the crop vigor map for the crop field utilizes a first machine vision algorithm executable by the hardware processor.
 4. The system of claim 3, wherein the machine vision algorithm comprises one or more deep learning neural networks, and wherein the deep learning neural networks are trained on ground truth data comprising geospatial image data of one or more sample crop fields and one or more crop vigor maps for the one or more sample crop fields.
 5. The system of claim 1, further comprising program code to: receive one or more crop field parameters, wherein the one or more crop field parameters are used in combination with the crop vigor map in the risk model to predict the pest susceptibility.
 6. The system of claim 5, wherein the one or more crop field parameters are determined from the geospatial image data utilizing a second machine vision algorithm executable by the hardware processor.
 7. The system of claim 5, wherein the one or more crop field parameters are machine data received from agricultural equipment.
 8. The system of claim 5, wherein the one or more crop field parameters comprise crop row spacing of the crop field, and wherein the crop row spacing is estimated from the geospatial image data using a third machine vision algorithm and/or received from machine data from one or more agricultural equipment.
 9. The system of claim 5, wherein the one or more crop field parameters comprise an irrigation status of the crop field, wherein the irrigation status is estimated using geospatial image data and/or received from machine data from one or more irrigation equipment.
 10. The system of claim 5, wherein the one or more crop field parameters comprise a degree of canopy closure of the crop field, wherein the degree of canopy closure is estimated from the geospatial image data using the first machine vision algorithm.
 11. The system of claim 5, wherein the one or more crop field parameters comprise a crop stage of the crop field, and wherein the crop stage is determined from physical observation and/or one or more crop models.
 12. The system of claim 1, wherein the risk model is a machine learning algorithm executable by the hardware processor, and wherein the machine learning algorithm is trained on ground truth data comprising one or more sample pest data points and one or more sample crop vigor maps for one or more sample crop fields.
 13. The system of claim 12, wherein the machine learning algorithm comprises one or more of a linear regressor, a nonlinear regressor, a random forest algorithm, and a neural network.
 14. The system of claim 1, wherein the crop pest is selected from the group consisting of crop diseases, insects, weeds, and plant pathogens.
 15. The system of claim 1, wherein the geospatial image data is selected from the group consisting of aerial imagery, satellite imagery, and unmanned aircraft system (UAS) imagery, and wherein the one or more sensors are infrared cameras.
 16. The system of claim 1, wherein the one or more sensors are located on a machine selected from the group consisting of an unmanned aerial vehicle (UAV), an unmanned aircraft system (UAS), an aircraft, a satellite, and a field equipment.
 17. The system of claim 1, wherein the microclimate data comprises a microclimate map, and wherein the microclimate map is generated from the group consisting of one or more in-field measurements, one or more aerial measurements, one or more satellite measurements, one or more drone measurements, and one or more microclimate models.
 18. The system of claim 1, further comprising program code to: generate a treatment plan based on the pest susceptibility map, wherein the treatment plan comprises an agricultural management technique, comprising application of one or more agricultural chemicals, to prevent outbreak or control propagation of one or more crop pests.
 19. The system of claim 1, further comprising program code to: receive price information for a crop growing in the crop field, a cost information for one or more agricultural management techniques, and an anticipated efficacy for the one or more agricultural management techniques; and generate an anticipated return on investment (ROI) based on the price and cost information and the anticipated efficacy.
 20. A computer-implemented method for predicting a pest susceptibility, the computer-implemented method executable by a hardware processor, the method comprising: receiving geospatial image data of a crop field from one or more sensors, wherein the geospatial image data of the crop field is geocoded by longitude and latitude coordinates; receiving microclimate data of the crop field, wherein the microclimate data comprises locally variable environmental conditions in one or more portions of the crop field; determining a crop vigor map for the crop field from the geospatial image data, wherein the crop vigor map comprises a crop vigor index on the one or more portions of the crop field, and wherein the crop vigor index is a numerical index based on a foliage volume, density, layout, and/or health status of the crop in the crop field; and generating a pest susceptibility map utilizing a risk model based on the crop vigor map and the microclimate data, wherein the pest susceptibility map comprises a pest susceptibility index geocoded by latitude and longitude coordinates, and wherein the pest susceptibility index is a measure of a susceptibility of a crop in the crop field to one or more crop pests. 